Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep track of remote xlog and basebackup #367

Closed
wants to merge 18 commits into from

Conversation

Vampouille
Copy link

The purpose of this PR is to keep tracks of remote files uploaded. As mentioned in #366, with a list of xlog on remote storage, we can compute several new metrics and check if there is gap in the xlog sequence. This PR is a based for another PR that will add new metrics as described in #366.

This PR mainly adds 2 attributes on PGHoard class:

  • remote_xlog
  • remote_basebackup

and code to keep those attributes up to date:

  • First run of handle_site() will fetch xlogs and basebackups from remote storage and populates remote_xlog and remote_basebackup attributes.
  • On successful tranfert, new xlog or basebackup will be added to remote_xlog and remote_basebackup attributes.
  • When deleting xlog or basebackup, remote_xlog or remote_basebackup attributes will be updated.

Additionally, this PR merge remote_basebackup attribute with state["backup_sites"][site]["basebackups"]:

self.state["backup_sites"][site]["basebackups"] = self.remote_basebackup[site]

Vampouille added a commit to camptocamp/pghoard that referenced this pull request Oct 2, 2019
@Vampouille
Copy link
Author

@Ormod Can you take a look at this merge request ? As discussed in #366, I want to keep track of remote files uploaded. So I added some attributes to store remote files and expose metrics:

  • total wal segments
  • continious wal segments
  • valid base backup count
  • missing wal segments
  • missing wal segments at end
    This PR might impact performances on database with high traffic so I would like to have your point of view.

Vampouille added a commit to camptocamp/pghoard that referenced this pull request Mar 11, 2020
@Ormod Ormod closed this Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants